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ABSTRACT 

In  an  earlier  report  I4rJ  a  consistent  estimation  scheme 
was  given  for  Gaussian  Markov  random  field  models.  In  this 
report  we  consider  some  statistical  properties  of  the  resulting 
estimate.  Specifically,  we  derive  an  expression  for  the  asym¬ 
ptotic  mean  square  error  of  the  estimate  for  a  general  model 
and  compare  the  efficiency  of  this  estimate  with  the  popular 
coding  estimate  for  a  simple  first  order  isotropic  model.  __ . 
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1 .  Introduction 

Markov  random  field  (MRF)  models  are  of  much  interest  in 
image  analysis  and  processing.  For  instance,  they  have  been 
used  in  texture  analysis  [2,3]  and  image  restoration  [4-6]. 

They  are  also  of  interest  in  the  analysis  of  field  data  [7] . 

MRF  models  characterize  the  special  nature  of  the  statistical 
dependency  of  intensity  levels  over  a  neighborhood  in  an  image. 

If  y(s)  denotes  the  intensity  level  at  location  s,  then  y(s) 
is  written  as  a  linear  combination  of  intensity  levels  {y(s+r), 
r€N}  (where  N  is  known  as  the  neighbor  set  of  dependence)  and 
additive  noise.  The  members  of  set  N  are  pairs  of  integers 
(k,£)  not  including  (0,0).  For  example,  the  first  order  MRF 
model  results  when  N=  { (0/1) # (1,0) , (0,-1) , (—1,0) }  and  the  second 
order  MRF  model  results  when  N=  { (-1 , 0) , (1 , 0 ) , (0 , -1 ) , (0 , 1 ) , (-1 , -1 ) , 
(-1,1) ,  (1,-1) , (1,1) } .  Each  MRF  model  is  characterized  by  a  set  of 
linear  weights  and  the  variance  of  the  additive  noise. 

Suppose  we  are  given  an  array  of  intensity  level  variations 
{y(s),sf^},  ft={s:(i,j),  lSi,jSM}  and  we  are  interested  in  fitting 
a  Gaussian  MRF  model  to  this  data.  We  also  assume  that  the  spec¬ 
ific  structure  of  the  MRF  model  characterized  by  the  neighbor  set 
N  is  known.  The  problem  of  estimating  N  has  been  considered  else¬ 
where  [1,8].  Three  estimation  methods,  namely  the  coding  method 
[7],  the  maximum  likelihood  (ML)  method,  and  the  method  in  [9] 
can  be  used  to  obtain  estimates  of  parameters  characterizing  a 
MRF  model.  It  is  the  intent  of  this  paper  to  analyze  in  some 
detail  the  statistical  properties  of  the  estimate  in  [1,8]. 


Specifically,  we  shall  derive  an  expression  for  the  asymptotic 
mean  square  error  of  the  estimate  for  an  arbitrary  MRF  model. 
We  evaluate  this  expression  for  a  simple  first  order  isotropic 
MRF  model  and  compare  the  efficiency  of  the  estimate  with  the 
popular  coding  estimate. 
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2.  Estimates  and  their  statistical  properties  in  MRF  models 
2.1  Model  Representation 


Assume  that  the  given  image  data  (y(s)}  obeys  the  MRF  model 
in  (2.1),  with  the  associated  neighbor  set  N: 


y(s)=  E  0  y(s+r)+e(s),sffl,  (2.1) 

r€N  r 

In  (2.1)  the  neighbor  set  N  is  symmetric,  i.e.,  if  r€N  then 
-rCN.  Further,  the  coefficients  satisfy  the  constraint  0r=6_r. 
The  stationary  noise  sequence  {e(s)}  is  partly  characterized  by 


E(e(s) |y(r))=0  Vs*r 


(2.2) 


Using  (2.1)  and  (2.2)  one  can  prove  that  the  noise  sequence 
{e(s)>  is  correlated  with  the  correlation  structure 

E (e (s)  e(r))*y,  s*r 


=-9s_rv, (s-r) €N  (2.3) 

=  0  otherwise 

It  can  be  shown  by  methods  similar  to  [10,11]  that  an  observation 
y(s)  obeying  (2.1)  with  the  conditions  (2.3)  indeed  satisfies 
the  Markov  condition 

p(y(s)  |  y(r) ,r€n,r^s) 

=  p(y  (s)|  y (s+t) , t€N) 

We  shall  characterize  the  neighbor  set  for  the  MRF  model  using 
the  set  Ng,  which  includes  members  from  the  non-symmetric  half 
of  N,  i.e.,  if  s€Ng  then  -s?Ng  and  N  =  {sss€Ng)u{-s:s€Ns} .  A 
sufficient  condition  to  ensure  stationarity  is  [7] 

Si:9k,Jlzl  z2  <  1  whenever  |z1|  ■  |  z2  |  «  1 


2 . 2  A  consistent  estimation  scheme  [1/8] : 
Consider  the  estimates 


0*  =  [E  q (s)  qT (s) ] -1 (E  q (s)  y(s))  (2.4) 

fi-'-  ~  fi  j.  " 

and 

v*  =  E  (y (s) -  9  *Tq (s)  )2  (2.5) 

M2  fi  j.  " 


where 


n  _  =  n  - 


and 


fi  =  {s  =  (i,j):  s 6fi  and  (s+r)/fi 

Jj 

for  at  least  one  r€N} 

We  now  state  a  theorem  regarding  the  consistency  of  the  estimate 

9*  and  give  an  expression  for  the  asymptotic  variance  of  the 

* 

estimate  0  .  An  expression  for  the  asymptotic  variance  of  0*  for 
an  isotropic  MRF  model  with  Ng  =  {  (0,1) , (1,0) }  is  also  given. 


Theorem  1 :  Let  y(s),  s* fi  be  the  set  of  observations  obeying 
the  MRF  model  (2.1).  Then 

(i)  The  estimate  0*  is  asymptotically  consistent. 

(ii)  The  asymptotic  covariance  matrix  of  0*  is 
E(9-9*)  (e-e*)T=  -Jv  Q-1  +  2v2(QTQ)_1 

m2  ~  ~ 

(2.6) 


£5 


-1  E  E 


s  r  ns-r)!:,,1?'1'1' 
(s-r  )€N 


where 


(iii)  For  the  isotropic  conditional  model  with  Ng  = 
{(0,1), (1,0)},  the  asymptotic  expected  mean  square  error  is 


,  2e2(l-46a.  ft)2 

j — 


e (8  -  e*) 


(2.7) 


4M  a 


1,0 


ll,0 


=  cov(y(s) ,y (s+(l,0) ) ) 
cov (y  (s)) 


(2.8) 


The  elements  of  matrices  Q  and  T  are  functions  of  normalized 

~  ~r ,  s 

autocorrelation  coefficients  The  proof  is  given  in  Appen¬ 

dix  I. 

Although  0*  is  a  consistent  estimate  of  0,  it  is  not  very 
efficient.  We  compare  the  efficiency  of  this  estimate  with 
efficiencies  of  the  coding  estimate  for  a  simple  MRF  model  with 
Ng  =  { (0,1), (1,0) } .  The  exact  ML  estimate  is  obtained  by  assuming 
a  toroidal  lattice  representation  for  y(s)  and  maximizing  the 
resulting  log  likelihood  function  by  using  the  Newton-Raphson 
procedure.  Equation  (2.4)  can  also  be  used  for  toroidal  lattice 
representation  by  summing  over H  instead  of  n  .  The  resulting 
difference  in  the  numerical  value  is  negligible  for  sufficiently 
large  M. 


2.3  Comparison  of  estimates 


We  compare  the  asymptotic  variance  of  the  estimate  (2.4) 
with  the  asymptotic  variances  of  the  coding  estimate  and  ML 
estimate  for  the  isotropic  conditional  models  with  Ng  =  {  (0,1), ( 
From  [9] ,  the  asymptotic  variance  of  the  coding  estimate  6C  is 

6(1-400,  J 


M2var  (ec)  -  -ra-— ^ 


(2.9) 


1,0 


Also  from  [9] ,  the  variance  of  the  ML  estimate  8„_  is 

MX* 


where 


Var(0ML)  = 


0.5 


(2.10) 


M  (I (9 )  -  4VJq (0) ) 


X(0)  =  1  jj  (cos  x+cos  y)2dxdy 

4ir2  0  (1-20  (cos  x  +cos  y)  ) 


and 


2  7T 

/ax  _  1  tr  cos (sx+ty)dxdy 

vst(0)  -  ^2  Ji  (1-20 (cos  x+  cos  y) 


Tabulated  values  of  V1Q(0),  Q  and  1(0)  are  available  in  [9] 
for  different  values  of  0.  Using  these  values,  and  (2.7),  (2.9 
(2.10),  Columns  2-4  of  Table  I  are  computed.  The  asymptotic 
efficiencies  in  columns  5  and  6  are  defined  by 


eff (0C) 


Var<»  ML> 
Var(9ci 


and 

Var<9  ML> 
Var ( 6  *) 


1,0)}  . 


and 


1 


eff (0*) 


It  is  evident  that  the  estimate  0*  computed  using  (2.4)  is 
more  efficient  than  the  coding  estimate  but  is  not  as  good 
as  the  ML  estimate.  Note  that  column  5  is  available  in  [9] . 
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Appendix  I 


proof  of  Theorem  T_: 
i)  We  have  from  (2.4) 


9*  =  C^q(s)qT(s)]'1(  £q(s)y(s)> 


(1) 


Substituting  for  y(s)  from  (2.1)  and  simplifying,  we  have 


[^q(s)qT(s)](6*-«)  =  ^q(s)e(s) 


(2) 


Since  E(q(s)e(s))  =  0  by  (2.2)  and  C  ^q(s)qT(s)D  is  a  positive  definite 


matrix,  the  consistency  of  the  estimate  e  follows. 


C*3 


ii)  To  make  our  calculations  easy  we  assume  from  now  on  thate(s)is  normally 
distributed.  Multiplying  (2)  by  its  transpose  and  taking  expectations, 
we  have 


EC]T)q($)qT(s)  (e*-9)(e*~8  ]r(^3q(s)ql(s)),3 


T,_\ %  T. 


=  EC  q(s)e(s)  53  (q(r)e(r))^3 


(3) 


=  535ZE*3<s)e*s))  E(qT(r)e(r)> 
s  r 

+5353  E(g(s)e(r>>  E(gT(r)e(s>) 
s  r 


(4) 


♦££E(e(s>e<r))  E(g(r)9T(s)> 


s  r 


s  i  +  ii  +  Ills  where 


I  =  0  , 


by  using  (2.2); 


11 


=  2  I  ,  by  using  (2.2)/ 


(5) 


(6) 


(7) 


mxm 


III  =  v  ]T)  E(q(s)qT(s)) 
s 


-  »  EE  \,-r>  E(9tr>9T<s» 

r  s 

(s-r) cN 


for  large  values  of  M 

q(s)qT(s)  =  Q  +  n(M), 
M  s  ~ 


where  n(M)  is  such  that 


ECn  CM)) 


■"(=■) 


Using  (11)  we  obtain 
(LHS  of  (3))/M4  = 


EC(Q  ♦  n(M))(e*-e)(e*-e)T(Q  +  n(M))T] 


*  Q  E<e*-0)(0*-0)T  qt  +  0(1/M2), 


Substitution  of  (10)  and  (13)  into  (3)  yields 


E(0*-0)(0*-0)T  =  Cv  Q-1  +  2v2(Q2)_1 
'  '  ~  '  f-T 


72  i'  EE  Ir,s<S>-'  *  «(1/H  N 

n  s  r 

(s-r) cN 


Q.E.O. 


c)  For  the  isotropic  conditional  model  with  =  <(0,1)> (1,0)>,  (14) 


reduces  to 


E(0*-e)2  =  ^ - \ - =•  C4v* 

FT  (E(q^(s))^ 


(15) 


♦  vE(qz(s))  -  8v  £53  E(q(s)q(r))3 

s  r 

(s-r) cN 


where 


q(s)  =  y(s+(0,1))  +  yCs+<0,-1>> 


+  y(s+(1/0)>  +  y(s+<-1,0>> 


Let 


Yk  L  =  ECy(s)y(s+(k,l))3 


Note 


1 


(16) 


(17) 


Yk,i  =  T-k  ri  and  Yi,k  =  Yk,i  <18) 

Express  the  higher  order  correlations  Yg  y  Y2  g  and  Yj  g  in 


Y0,0  "  Y1,1 


(22) 


y 


2,0 


1 

•  Y1,0 


r1,-1 


yi,-2  =  ?e  Yi,-i  ~  Yi,o  <23) 

Equations  (19-23)  can  be  obtained  by  multiplying  y(s)  by  appropriately 
shifted  y(s+(k,l))  and  taking  expectations. 

Consider  the  various  terms  in  (15). 


ECq^(s)) 


(24) 


EE  E(q(s)q(r))  =  4C9t1#q  +  yJQ  ♦  3Y2J  +  3Y,t_2l 
(s-r)eN 

*  tC‘T1,0  *  ^2  Y1,0  -  T  T0,03 

Substitution  of  (24)  and  (25)  in  (15)  yields 


E(e*-e)2  =  1-  — —  C4v2 

"  ’<0 


which  on  using  (19)  and  q  = 


*  2  2e2<l-4ea1  n)2" 

»  -«>z  = - liM — 


E(9-9)  = 


+  4VYo,o  - 

T1,0/T1,1  91ves 


(25) 


References 


1)  R.  Chellappa,  "Fitting  Markov  random  field  models  to 
images,"  Technical  Report  TR-994,  Computer  Science 
Center,  University  of  Maryland,  College  Park,  Maryland, 
20742,  Jan.  1981. 

2)  M.  Hassner  and  J.  Sklansky,  "The  use  of  Markov  random 
fields  as  models  of  textures,"  Computer  Graphics  Image 
Proc.,  Vol.  12,  pp.  357-370,  April  1980. 

3)  G.R.  Cross,  "Markov  random  field  texture  models,"  Technical 
Report  No.  80-02,  Department  of  Computer  Science,  Michigan 
State  University,  East  Lansing,  Michigan,  1980. 

4)  R.L.  Kashyap  and  R.  Chellappa,  "Image  restoration  using 
random  fields,"  Proc.  of  the  Eighteenth  Annual  Allerton 
Conf.  on  Comm.,  Control  and  Computing,  University  of 
Illinois,  Urbana,  Ill.,  pp.  856-965,  Oct.  1980. 

5)  J.W.  Woods,  "Markov  image  modeling,"  IEEE  Trans.  Automat. 
Contr.,  Vol.  AC-23,  pp.  846-850,  Oct.  1978. 

6)  A.K.  Jain,  "A  fast  Karhunen-Loeve  transform  for  digital 
restoration  of  images  degraded  by  white  and  colored  noise/' 
IEEE  Trans.  Computers,  Vol.  C-26,  pp.  560-571,  June  1977. 

7)  J.E.  Besag,  "Spatial  interaction  and  statistical  analysis 
of  lattice  systems,"  J.  Royal  Stat.  Soc.,  Ser.  B,  Vol.  B-36 
pp.  192-236,  1974. 

8)  R.L.  Kashyap  and  R.  Chellappa,  "Estimation  and  choice  of 
neighbors  in  spatial  interaction  models  of  images"  (sub¬ 
mitted  for  publication) . 

9)  P.A.P.  Moran  and  J.E.  Besag,  "On  the  estimation  and  testing 
of  spatial  interaction  in  Gaussian  lattices,"  Biometrika, 
Vol.  62,  pp.  555-562,  1975. 

10)  J.W.  Woods,  "Two-dimensional  discrete  Markov  random  fields, 
IEEE  Trans.  Information  Theory,  Vol.  18,  pp.  232-240,  March 
1972. 

11)  R.L.  Kashyap,  "Random  field  models  on  finite  lattices  for 
finite  images"  (presented  at  the  Symposium  on  Information 
Sciences,  Baltimore,  MD,  March  ,  1981). 


SECURITY  CLASSIFICATION  of  THIS  FACE  (Whit  Dio  Entered) 


REPORT  DOCUMENTATION  PAGE 


refort  number 


'.Vi  M. 


AFOSR-TR-  8  1  -0  5  56 


1 


READ  INSTRUCTIONS 
BEFORE  COMPLETING  FORM 


>•  recipient's  catalog  number 


«.  title  (ltd  Submit) 


S.  TYPE  OF  REPORT  S  FERIOO  COVERED 


ON  AN  ESTIMATION  SCHEME  FOR  GAUSS  MARKOV  Technical 

RANDOM  FIELD  MODELS  s.  performing  oro.  report  number 

TR- 1040 


s.  contract  or  grant  NUMBER?*) 


7.  author?*) 


R.  Chellappa 


AFOSR-7 7-3271 


s.  performing  organization  name  and  address 

Computer  Vision  Laboratory,  Computer 
Science  Center,  University  of  Maryland 
College  Park,  MD  20742 


it.  controlling  office  name  ano  address 

Math.  &  Info.  Sciences,  AFOSR/NM 
Bolling  AFB 

Washington,  DC  20332  _ 


4.  monitoring  AGENCY  NAME  t  AOORESS (II  dllltrltl  Iron  Controlling  Ollleo) 


10.  PROGRAM  ELEMENT.  PROJECT.  TASK 
AREA  *  WORK  UNIT  NUMBERS 

Q'/ZCrZ/^ 

J2  .?  O 


12.  REPORT  DATE 

April  1981 


IS.  NUMBER  OF  PAGES 

15 


IS.  SECURITY  CLASS,  (ol  thlt  report) 

Unclassified 


IS*.  OECL ASSI FICATION/  DOWNGRADING 
SCHEDULE 


DISTRIBUTION  STATEMENT  ?•«  thlt  Report) 


Approved  for  public  release;  distribution  unlimited. 


17.  DISTRIBUTION  STATEMENT  (ol  tho  mbotroct  ontorod  In  Block  70.  II  dlllortnt  liom  Report) 


IS.  KEY  WOROS  (Continue  on  reroree  tide  II  noceetery  end  Identity  by  block  number) 


Image  processing 
Image  models 
Random  fields 
Estimation 


20.  ABSTRACT  (Continue  on  rente  tide  II  nocoetoey  end  Identity  by  block  number) 

In  an  earlier  report  [1]  a  consistent  estimation  scheme  was  given 
for  Gaussian  Markov  random  field  models.  In  this  report  we  con- 
side  some  statistical  properties  of  the  resulting  estimate. 
Specifically,  we  derive  an  expression  for  the  asymptotic  mean 
square  error  of  the  estimate  for  a  general  model  and  compare  the 
efficiency  of  this  estimate  with  the  popular  coding  estimate  for 
a  simple  first  order  isotropic  model. 


DD  ,X*ts  1473 


EDITION  OF  I  NOV  ••  IS  OBSOLETE 


SECURITY  CLASSIFICATION  OF  This  PAGE  ?W>i*i»  Die  Entered) 


